Statistical Query Algorithms for Mean Vector Estimation and Stochastic Convex Optimization

Stochastic convex optimization, by which the objective is expectation of a random function, an important and widely used method with numerous applications in machine learning, statistics, operations research, other areas. We study complexity stochastic optimization given only statistical query (SQ) access to function. show that well-known popular first-order iterative methods can be implemented using queries. For many cases interest, we derive nearly matching upper lower bounds on estimation (sample) complexity, including linear most general setting. then present several consequences for differential privacy, proving concrete power optimization–based methods. The key ingredient our work SQ algorithms estimating mean vector distribution over vectors supported body R d . This natural problem has not been previously studied, solutions get substantially improved versions Perceptron online learning halfspaces.